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Introduction 

The Job Training Partnership Act is designed to be a performance- 
driven program. Performance standards are the tools that are used 
to assure that job training programs are a productive investment in 
human capital. 

To fulfill the Secretary's mandate and assure that the goals of the 
Job Training Partnership Act are achieved, the U.S. Department of 
Labor has selected twelve performance measures, established a 
national standard for each measure, and designed a national 
adjustment methodology for adapting the national standards to local 
conditions. States must set standards for local service delivery areas 
for eight of the twelve, using either DOL's adjustment methodology 
or their own adjustment methodology established within 
paran ters set by the Secretary. States must also determine 
whether these standards have been met, provide technical 
assistance, reward performance, and impose sanctions when 
standards are not met for two years. Governors must also set an 
entered employment rate standard for Title III formula funded 
programs and establish a goal for Title in wage at placement. 
Service delivery areas must formulate policies and operate 
programs that will meet or exceed the state-established performance 
standards, but they are encouraged to seek further a<^ustments if 
local conditions warrant. 

Clearly, the most challenging aspects of implementing the new 
performance standards system fall to professionals at the local level. 
It is there that policies are made and programs are shaped. The 
purpose of this guide is to provide local policy makers and program 
managers with an approach for using performance standards as a 
tool for reviewing and improving performance at the local level. The 
guide is keyed to the PY 88 DOL adjustment methodology for 
activities beginning July 1, 1988. Earlier versions of this guide, 
which include the PY 84 through PY 87 models, are available 
through the National Alliance of Business. An outline of the guide 
follows. 

Section I: Overview of Performance Standards - This section 
discusses the evolution of and rationale supporting performance 
standards. 

Section II: IdentiHcation of Local Performance - This section 
details a systematic approach to obtaining comprehensive baseline 
data on local performance, an analysis of the performance of specific 
activities and contractors within the SDA, an analysis of perfor* 
mance trends, and a comparison with other SDAs. 



ERLC 



1 7 



Section III: Strategies for Improving Performance - This 
section describes two approaches for improving performance: Hrst, 
the Oversight Response, which identiHes probable causes of poor 
performance, and second, the Planning Response, which provides 
tools to examine new strategies and approaches that will improve 
performance. 



Section IV: Conclusion - This section summarizes the key issues 
related to planning and managing with performance standards. 

Readers are cautioned not to rely solely on performance standards to 
plan, manage, and evaluate JTPA; performance standards are only 
one measure of the effectiveness of JTPA. The mission established 
oy the private industry council (PIC) and local elected officials may 
make other goals as significant as achieving the performance 
standards. For example, the use of JTPA in effecting institutional 
change at the local level or the use of JTPA in promoting job 
creation and economic development activities could be as important 
as local performance measures and must be considered in tiie self- 
evaluation process. 

Many of the reports and worksheets presented in the guide require 
routine, yet time-consuming, computations using local participant 
and financial data. A software option is available for microcomputer 
users that speeds up the analysis and allows for graphic 
presentation of data. This program operates on MS DOS 2.0 and 
higher and requires an IBM PC, XT, AT or true compatible with a 
minimum of 256 K. The cost of the software is $399.00 with future 
year updates priced at $50.00. For more information on the 
software, contact the Employment and Training Institute at 1-800- 
932-0085. 

The software option includes the following reports and worksheets 
which are referenced in this guide: 

Title Worksheet Number 

Performance Status Report 1 
Performance Status Graph 2 
Analysis of Enrollments and Expenditures 3 
Performance of Terminees by Characteristic - Adult 4 
Performance of Terminees by Characteristic Youth 5 
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Analysis of Programs and Contractors • Adult 6 

Analysis of Programs and Contractors • Youth 7 

Model A4]usted Worksheet for Analyzing Client 
Characteristic Performance - Adult 13 

Model A4]usted Worksheet for Analyzing Client 
Characteristic Performance - Youth 14 

Model A4]usted Worksheet for Analyzi:^g Client 
Characteristic Performance - Adult Post Program 15 

Model Adjusted Worksheet for Analyzing Client 
Characteristic Performance - Welfare 16 
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Section I: 



Overview of Performance Standards 



Throughout the 20-year evolution of Federal job training programs 
the government, local policy makers, and practitioners have agreed 
on the need to assess program performance. An equitable method of 
assessing these local programs, however, was a complex issue. Each 
community has a unique set of circumstances which directly affects 
its ability to operate effective job training programs. A program in 
an urban setting with a high unemployment rate serving a high 
percentage of "hard to serve" individuals may be performing well if 
40 percent of trainees are placed in jobs. A similar program in a 
suburban setting with a low unemployment rate serving a low 
percentage of "hard to serve" individuals may be performing poorly 
if only 40 percent of trainees are placed in jobs. Since no standard 
method of making ac^ustments for local variables was available, the 
expected performance of past programs was determined by 
negotiations between local program operators and representatives of 
the Department of Labor. 

Performance standards under JTPA are different: They allow for 
local variations in measuring the effectiveness of job training 
programs. Thus, performance standards have become an 
"adjustment system" in which the actual local standard reflects such 
factors as who is served and local economic conditions. For example, 
an SDA with a low unemployment rate serving a low percentage of 
difHcult-to-place trainees will have comparatively high standards of 
performance. In contrast, an SDA with high unemployment serving 
a high percentage of difficult-to-place trainees will have relatively 
low standards of performance. If the model works as intended, the 
second SDA would not be penalized because of difficult local 
conditions; its standards would be adjusted downward. 

The speciHc methodology incorporated in perform^ince standards 
has evolved over the last six years. For the next two year planning 
cycle (PY 88 - PY 89) five new measures have been added to the 
original seven performance standards. These are: a youth 
employability enhancement measure and four post-program follow- 
up measures designed to assess the long-term impact of JTPA 
services. Governors are required to establish numerical 
performance standards in each SDA for at least eight of the 12 
Federal standards. Governors are also required to establish a 
numerical standard for the entered employment rate in Title m 
(Dislocated Worker) Programs and are encouraged to establish an 
average wage at placement goal for Title m terminees. 
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The Federal strategy for meeting performance management goals 
includes encouraging governors to use the authority granted to 
them under JTPA to further adjust SDA performance standards to 
address broader performance management concerns. THE GUIDE 
FOR SETTING JfPA TITLE IhA PERFORMANCE STANDARDS FOR 
PY 88 . published by the U.S. Department of Labor is an essential 
resource for seeking such adjustments. 

The performance standards system is the first attempt to establish a 
national system for judging program performance that allows 
adjustment for the local factors that affect performance. In 
developing its adjustment approach, the Department of Labor 
analyzed data on prior programs and chose variables which were 
found to have a significant impact on performance. Each variable 
was weighted according to its average impact on performance. For 
example, by plotting the data from hundreds of communities, it was 
found that serving more dropouts resulted in fewer placements. The 
influence of this factor was determined mathematically and a 
corresponding weight was assigned. The current methodology is an 
extension of this simple example. Ultimately 23 variables were 
found to affect performance significantly and are included in the 
model. Several critical policy issues were addressed in the 
establishment of the current methodology. Many of these issues 
included technical items such as statistical modeling, selection of 
data bases from CETA and JTPA, and tolerance level adjustments. 
Three of the most significant areas addressed deal with the selection 
of the performance measures, the establishment of national 
standards, and the identification of the local adjustment factors. A 
description of each of these issues follows. 



Specific 

Performance 

Measures 



Adult Measures 



The twelve performance measures selected by the Department of 
Labor include four measures for adult programs, four measures for 
youth programs and ibur measures for adult post-program 
performance under Title HA. Separate measures were established 
for the youth and adult populations based on the differing needs and 
historical results for each population. 

• Entered Employment Rate - the number of adults who entered 
emplojonent at termination as a percentage of the total number 
of adults who terminated. 



Welfare Entered Employment Rate - the number of adult 
welfare recipienti^ who entered employment at termination as a 
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percentage of the total number of adult welfare recipients who 
terminated. 

• Average Wage at Placement - average hourly wage for all adults 
who entered employment at the time of termination. 

• Cost per Entered Employment - total expenditures for adults 
divided by the number of adults who entered emplojonent. 

Youth Measures • Entered Employment Rate - the number of youth who entered 

employment as a percentage of the number of youth who 
terminated. 

• Positive Termination Rate - the number of youth who had a 
positive termination as a percentage of all youth who termi- 
nated. Positive terminations include entered employment, 
obtained employability enhancements, or attained PiC-approved 
youth competencies. 

• Cost per Positive Termination - total expenditures for youth 
divided by the number of youth who had a positive termination. 

• Employability Enhancement Rate - the number of youth who 
attained one of the employability enhancements whether or not 
they also obtained a job as a percentage of the total number of 
youth who terminated. Youth Employability Enhancements 
include: 

a. Attained PIC-Recognized Youth Emplojrment Conipetencies. 

b. Entered Non-Title n Training 

c. Completed Major Level of Education 

d. Completed Program Objectives (14-15 year olds). 

Post- Program • Follow-up Employment Rate - the number of Adult respondents 
Measures who were employed during the 13th Week after program 

termination as a percentage of the total of all respondents 
(terminees who completed follow-up interview) 

• Welfare Follow-up Employment Rate - the number of Adult 
Welfare respondents who were unemployed during the 13th 
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week after progran?. termination as a percentage of the total of all 
adult welfare respondents. 



Weekly earnings at Follow-up total gross weekly earnings of 
respondents who were employed at the 13th week divided by the 
total number of respondents empkyed at the 13th week. 

Weeks Worked in the Follow-up Period - total number of weeks 
worked in the 13 week follow-up period for all respondents 
divided by the total number of respondents. 



Performance The Secretary of Labor established national performance standards 

Standards (specific numerical le /els) for each of the twelve measures based on 

Title IIA an analysis of prior accomplishments. The Secretary's standards 

have been set at a point . which 75% of the SDAs are expected to 
exceed based on previous performance. This differs from prior years 
when meeting the Secretary's standards simply represented average 
performance. The national performance standards are as follows: 

Adult Standards 

Entered Employment Rate 68% 

Welfare Entered Employment Rate 56% 

Average Wage at Placement $4.95 

Cost per Entered Employment $4,500.00 

Youth Standards 

Entered Employment Rate 45% 

Positive Termination Rate 75% 

Cost per Positive Termination $4,900.00 

Employability Enhancement Rate 30% 

For Post Program 

Follow-up Employment Rate 60% 

Welfare Follow-up Employment Rate 50% 

Weeks Worked in the Follow-up Period 8 
Weekly Earnings of all Employed at 

Follow-up $177.00 
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Local 

Adjustment 
Factors 



The factors that are considered in adjusting the national standards 
for local conditions fall into three categories: the characteristics of 
those served, the length of training, and local economic conditions. 



Who Is Served 

The following trainee characteristics were found to affect 
performance significantly. They are either included in the law or 
are used in the adjustment model 



1. 


Female 


2. 


Agel4-]5 


3. 


Age 16-17 


4. 


Age 30 + 


5. 


Black 


6. 


Hispanic 


7. 


Asian/Pacific Islander 


8. 


Dropout 


9. 


Student 


10. 


Post high school attendee 


11. 


AFDC Recipient 


12. 


GA/RCA Recipient 


13. 


Unemployment compensation claimant 


14. 


Handicapped 


15. 


Offender 


16. 


Person not in labor force 


17. 


Unemployed 15 + weeks 



Length of Training /Percent Terminees 



Factors such as the length of training and percent of individuals 
in classroom and on-the-job (OJT) training were found to 
influence performance. However, for a variety of reasons, only 
one factor dealing with the length of the training program is 
included in the national model* 

This factor takes into account the percentage of participants who 
terminate during the program year and only influences the two 
performance measures dealing with tost. Experience has shown 
that programs carrying over more participants into the next 
program year tend to have higher costs than those programs that 
carry over fewer participants. 
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Local Economic Conditions 



Five factors which reflect local economic conditions are included 
in the model: local average wage, unemployment rate, 
population density, number of families below poverty level, and 
employee /resident worker ratio. 

The state may establish an SDA's performance standards using 
DOL's adjustment methodology or its own adjustment methodology 
established within the parameters set by DOL. This adjustment 
takes into consideration local SDA conditions and calculates a 
numerical performance level for each of the twelve measures. SDAs 
must plan and operate programs to meet the established standards 
or seek ac^ustments to the state-established standards when local 
conditions warrant. The ability to perform at or above the standard 
is largely dependent on the policy direction provided by the PIC and 
the planning and management skills of local administrators. 

The remainder of this guide provides an approach and some tools 
that will enable SDA/PIC staff to use the performance standards 
system as a ''point of departure'' for improving planning and 
management practices. 
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Section II: 



Identification of Local Performance 



In order to begin a serious examination of performance, an SDA 
must have accurate, complete program information available on a 
continuous basis. A sound management information system is 
essential. Without these resources a manager does not have the 
ability to determine the status of current activities and cannot begin 
the process of improving performance. This first level of review, 
which includes calci^lating current performance, baseline program 
review, and tro' and comparison analysis, allows the SDA to 
conduct a comprehensive analysis of overall performance. The 
following reports are required for this review: 

• State prepared SDA report 

• Locally prepared program reports 

Calculating Calculating your current performance enables you to take a snap- 

Current shot of local SDA performance and compare it to the twelve primary 

Performance JTPA performance standards, either as approved in your Job Train- 

ing Plan or as adjusted based on actual experience. Worksheet 1, 
Performance Status Report, ¥dll provide a report on current perfor- 
mance (actual performance) in relation to planned or reac^usted 
performance standards (expected performance). The reac^usted 
standards are obviously the most accurate way to analyze your 
performance since they base your standard on the actual character- 
istics of terminees, percent terminated, and current local economic 
data. For SDAs that wish to include the reac^usted standards. 
Worksheets 13, 14, and 15 provide the methodology to generate 
readjusted standards. Depending on which method has been 
selected by your state to calculate the Welfare Entered Employment 
Rate, you may also need Worksheet 16. The Performance Status 
Report indicates how close you are to meeting your standards. The 
required information may be available through existing systems 
either within your SDA or from the state. 

The following information is required to complete Worksheet 1. 

1. Number of adults terminated 

2. Number of adults entered employment 

3. Total adult expenditures 

4. Total youth expenditures 

5. Average hourly wage at placement (adults) 

6. Number of adult welfare recipients terminated 

7. Number of adult welfare recipients entered employment 

8. Number of youth terminations 

9. Number ofyouth entered employment 

10. Number of youth positive terminations 

11. Num'ber of youth employabili ty enhancements 
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12. Number of adults employed 13 weeks after program 
termination 

13. Number of Adult Welfare respondents employed 13 weeks after 
program termination 

14. Average number of weeks worked in the 13 week period after 
program termination for adults 

15. Average gross weekly earnings of all respondents employed at 
the 13th week after program termination 

16. Planned performance standards (Job Training Plan) or re- 
adjusted standards as calculated on Worksheets 13, 14, and 15 

If information on any category is unavailable, it is not possible to 
identify your current level of performance in relation to the twelve 
key performance standards. Adjustments to your Management 
Information System may be required. 

Worksheets 1 and 2 are designed to provide an ongoing analysis 
capability, allowing SDAs to increase the frequency of performance 
analysis beyond that currently provided by state and local 
standardized reports. 

Baseline Baseline program review helps identify probable causes of poor per- 

Program Review formance through a comprehensive review of program outcomes and 

client characteristics. It begins the process of narrowing down the 
specific areas which influence performance by gathering data on 
program and subcontractor performance. 

This process begins with an initial review of total enrollments and 
expenditures. Deviations from plan in the areas of total served, 
youth service levels, and expenditure rates will have a dramatic 
impact on actual performance. Worksheet 3 provides a suggested 
format for this review. This review may indicate, for example, that 
although expenditures are 90 percent of plan, adult and youth 
enrollments are only 60 percent of plan. This situation may result 
in failure to meet the cost standards since this would increase the 
^cost per** outcome significantly. 

The next two steps in the baseline program review process involve a 
detailed analysis of actual performance: first by target groups and 
then by programs or contractors. Worksheets 4 and 5 will enable an 
SDA to analyze, by performance standard, youth and adult outcomes 
by terminee characteristics. Worksheets 6 and 7 will enable an SDA 
to analyze, by performance standard, youth and adult outcomes by 
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Worksheet 1 








PERFORMANCE STATUS REPORT 






Report Period: 






s 


Enter Current Values in Columns A, B, and C 


A 
Adult 






t: 


1. Number of Terminees 








c 


2. Number Placed 








T 


3. Positive Terminations 




HH 


1 


4. Employability Enhancements 




HH 


1 


5. Average Wage at Placement 






0 


6. Total Program Cost 








N 


7. F/U Employment Rate 










8. F/U Weeks Worked 






1 


9. F/U Weekly Earnings 








Enter Actual and Expected 
Performance and % of Standard 


A 
Adult 








Entered 


Actual Performance 










Employment 


Exoected Performance 










Rate 


% of Standard 










Cost per Entered 
Employment 


Actual Performance 








Expected Performance 






S 


% of Standard 








Average Wage 
at Placement 


Actual Performance 






c 


Exoected Performance 






c 


% of Standard 






c 


Positive 
Termination Rate 


Actual Performance ^^^^H 




Expected Performance ^^^^H 




■■■■■ 


T 


% of Standard ^^^^H 


■■■■1 


Cost per Positive 
Termination 


Actual Pefiormance ^^^^H 




1 


Exoected Performance ^^^^H 


■■■■■ 


1 


% of ^^H^l 


■■■■■ 


U 


Employability 
Enhancement 


Actual Performance ^^^^H 




Expected Performance ^^^^H 


■■■■ 


M 


Rate 


% of Standard ^^^H 


■■■■I 


Fol low-Up 
Employment 


Actual Performance 








Expected Performance 








Rate 


% of Standard 






2 


Fol low-Up 
Weeks Worked 


Actual Performance 








Expected Performance 








% of Standard 








Fol low-Up 
Weekly Earnings 


Actual Performance 








Expected Performance 








% of Standard 
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WorkshMt 2 Repftrt Pori 

PERFORMANCE STATUS GRAPH 

Transfer totals for expected performance (EP) and actual performance (AP) from 
Worksheet 1 PERFORMANCE STATUS REPORT to complete the following bar graphs. 

ADULT 


ENTERED 
EMPLOYMENT 


WELFARE ENTERED 
EMPLOYMENT 


COST PER ENTERED 
EMPLOYMENT 


WAGE AT 
PLACEMENT 


% EP AP 


% EP AP 


EP AP 


EP AP 










.100 


100 


7500 


6.00 


90 


90 


7000 
6500 


5.75 
5.50 


80 


80 


6000 


5.25 


70 


70 


5500 


5.00 


60 


60 


5000 
4500 


4.75 
4.50 


50 


50 


4000 


4.25 


40 


40 


3500 


4.00 


30 


30 


3000 
2500 


3.75 
3.50 


20 


20 


2000 


3.25 


10 


10 


1500 


3.00 


0 


0 


1000 
500 


2.75 


YOUTH 


ENTERED 


COST PER POSITIVE 


POSITIVE 


EMPLOYABILITY 


EMPLOYMENT 


TERMINATION 


TERMINATION 


ENHANCEMENT 


% EP AP 


EP AP 


% EP AP 


% EP AP 




$7000 






100 


6500 


100 


100 


90 


6000 
5500 


90 


90 


80 


5000 


80 


80 


70 


4500 


70 


70 


60 


4000 
3500 


60 


60 


50 


3000 


50 


50 


40 


2500 


40 


40 


30 


2000 
1500 


30 


30 


20 


1000 


20 


20 


10 


500 


10 


10 


0 




0 


0 
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WORKSHEETS 



Report Heriod 
ANALYSIS OF ENROLLMENTS AND EXPENDITURES 



12 Month 
Planned 



%of 
Total 



Planned 



%of 
Total 



Actual 



%of 
Total 



% of Planned 
for Period 



Enrollments (Total) 



Adult 



Youth 



Expenditures (Total) 



Adult 



Youth 



Administration 



Salary t Fringe 



Operating Costs 



Other 



Training 



Salary & Fringe 



Occupational Training 



Other Classroom Training 



OJT 



Special Programs 



Other 



Services 



Needs Based Payments 



Support Services 



ERjC Other 



WORKSHEET 4 



Report Period 

PERFORMANCE OF TERMINEES BY CHARACTERISTIC - ADULT 



CHARAaERISTICS 




ACTUAL PERFORMANCE 


ActualTerminees 


Entered 
Employment 
Rate 


Cost Per 
Entered 
Employment 


Average 
1 Wage 


Follow-Up 
Employment 
Rate 


Follow-Up 
Weeks 
Worked 


Folk>«v-Up 
Weekly 
Earnings 


# 


% 


Male 


















Female 


















Age 39 4 


















Black 


















Hispanic 


















Dropout 


















UC Claimant 


















Unempl. 154 


















Not in Labor Force 


















AFDC 


















uA/KCA 


















TOTAL 


















Others (Optional) 




















































JLMa_ 


C ""-^ 



















WORKSHEET 5 



Report Period 

PERFORMANCE OF TERMiNEES BY CHARACTERISTIC - YOUTH 



CHARACTERISTICS 



Male 



Actual Terminees 



ACTUAL PERFORMANCE 



Entered 
Employment 
Rate 



Positive 
Termination 
Rate 



Cost Per 
Positive 
Termination 



Employment 
Enhancement 
Rate 



Female 



Age 14-15 



Age 16-17 



White 



Black 



Hispanic 



Alaskan Native/American Indian 



Asian/Pacific Islander 



Dropout 



Student 



Not in Labor Force 



Post High School Attendee 



AFDC Recipient 



GA/RCA Recipient 



Handicapped 



Offender 



TOTAL 



24 



Others (Optional) 
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WORKSHEET 6 


Report Period 

ANALYSIS OF PROGRAMS/CONTRACTORS - ADULT 


Programs/Contracts 


PERFORMANCE DATA 


POST PROGRAM FOLLOW-UP 


Entered 
Employment 
Koie 


Welfare Entered 
Employment 

Kolc 


Cost Per 
Entered 
c in p 1 o y III c n I 


Average 
Wage at 
Placement 


Follow-up 
Employment 
Rate 


Follow*up 
Weeks 
worKcQ 


Follow*iJD 
Av^. Weekly 
earnings 


Occupational Training 

1. 

2. 

3. 

4. 

5. 

6. 
















Total Occupational Training 
















Other Classroom Training 
1. Job Finding Skills 
















2. Remedial Programs 
A. 
B. 
















3. Special Programs 
A. 

D. 

c. 
















OJT 






























2Z_ 


puTAL ALL PROGRAMS 

















WORKSHEET? 



Report Period 



ANALYSIS OF PROGRAMS/CONTRAaORS- YOUTH 



Programs/Contracts 


PERFORMANCE DATA 


Entered 
Employment 
Rate 


Positive 
Termination 
Rate 


Cost Per Positive 
Termination 


Employability Enhancement Rate 


Occupational Training 

1. 

2. 

3. 

4. 

5. 

6. 










Total Occupational Training 










Other Classroom Training 
1. Job Finding Skills 










2. Remedial Programs 
A. 
B. 










3. Special Programs 
A. 
B. 
C. 










OJT 










Other 










TOTAL ALL PROGRAMS 
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program or contractor. Taken together these reports will, in most 
cases, help isolate problem areas and provide a starting point for 
corrective action. This information will also be useful when you are 
planning new strategies to improve performance since you will 
know what programs work best, for what groups, and at what cost. 

Although the type of analysis that can be done using Worksheets 4-7 
can be very useful in pinpointing problem areas, SDAs are cautioned 
not to apply the SDA's overall expected performance levels to all 
programs and contractors or to specific target groups. 

Programs must be looked at in the context of varying client needs 
and service strategies. For example, if an SDA has an adult entered 
employment rate of 55 percent and a cost per entered employment of 
$5,800, the SDA might be tempted to identify programs and target 
groups performing below this level as **problem areas.** But, upon 
closer review, such programs may be exceeding model-based perfor- 
mance expectations based on the characteristics of participants 
being served and the type of training being provided. Similarly, a 
long-term program serving dropouts with an entered employment 
rate of 43 percent and a cost per entered employment of $6,400 may 
not necessarily reduce overall performance. 

Not all activities will produce outcomes consistent with the overall 
expected performance levels established for an SDA: some will 
produce higher results and some lower. The challenge is to 
construct a job training system that, in total, exceeds overall 
established performance levels. 

While analysis of the performance of specific program activities and 
subcontractors and the outcomes for specific client groups can be 
guided to a certain extent by the DOL model, DOL warns that a 
strict application of the model in these instances would be 
inappropriate. 

Trend and Once you have calculated your current level of performance and 

Comparison completed your baseline program review, you may want to conclude 

Analysis your overall performance analysis by examining performance trends 

and by comparing current performance with past SDA performance 
or with the performance of comparable SDAs. In order to complete 
both the comparison and trend steps you will need the following 
information: 



1 • Prior year's performance standards 

2. Performance reports (available from this section) 

3. Planned performance goals contained in the Job Training Plan 

4. Statewide performance reports (available through your state 
JTPA office) 

Trend Analysis Trend analysis allows the SDA to visualize, on a regular basis, 

performance variations over a longer period of time. With this type 
of analysis PIC members and SDA staff will easily be able to 
determine the direction in which the program is going. They can 
then take corrective action when a downward trend becomes 
apparent rather than waiting for performance to fall hopelessly 
below the standard. 

Performance may fluctuate at monthly or quarterly review points 
for legitimate rea&>3ns. Fluctuations in performance may have a 
number of explanations: closed entry courses, late program start- 
ups, or funding delays. Trend analysis may help pinpoint a 
potential problem area. It may indicate, for example, that your 
particular SDA has developed a trend of decreasing entered 
employment rates with increased cost per placement for a six month 
period. These trends would then be considered in analyzing SDA 
performance. 

Use Worksheets 8 and 9 to maintain an ongoing record of trends for 
each of the two performance standards groups, adult and youth. 
Worksheet 10 can be used to track each standard on a yearly or more 
frequent basis. 

Comparison Comparison analysis offers a broader perspective on **how you*re 

Analysis doing.** Comparison analysis provides you with the opportunity to 

compare your current performance with: 

• Past local SDA experience 

• Performance of comparable SDAs (economy, labor market, and 
unemployment rate similarities) 

• Statewide SDA performance 

The approved performance standards in your Job Training Plan 
represent minimal expectations. Many SDAs have established more 
ambitious performance goals or have planned incremental 
improvements in selected standards. You may find, for example, 
that although you have met all of your performance standards. 
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comparable SDAs are exceeding standards. Many SDAs are now 
competing for a portion of the "incentive** funds that are awarded by 
the states, thus, comparison analysis takes on new meaning. 

Use Worksheet 12 to record and compare your SDA performance 
with various jurisdictions within a state. This worksheet enables 
the SDA to pxamine the extent to which individual standards were 
met by comparable SDAs. For example, if the SDA standard for 
adult entered employment is 55 percent and the actual performance 
is 52 percent, the chart would display the percent of standard at 
94.5. 

The approach and reports detailed in this section complete the first 
level of performance review and should provide sufficient informa- 
tion for SDAs to determine priority areas for further investigation 
and improvement. This baseline data should be carefully reviewed 
with local policy makers and professional staff. Section III: 
Strategies for Improving Performance describes steps that can be 
taken to improve performance - first the Oversight Response which 
helps pinpoint reasons for poor performance and then the Planning 
Response which helps identify new strategies for improved results. 
In almost all cases, an SDA will be involved in some combination of 
the Oversight Response and the Planning Response. 
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WORKSHEETS Reoort Period 

ADULT TREND ANALYSIS 

Complete the bottom portion of the worksheet and graph the quarterly/monthly 
trends in performance standards. 


% of Standard 


Period Ending 


Period Ending 


Period Ending 


Period Ending 


150% 
140% 

130% 

120% 

110% 










90% 
80% 
70% 
60% 
50% 
40% 












Performance 
Standards 


Legend 


% of Standard 


Entered Emp. 












Cost Per E E. 












Welfare E.E. 












Wage at PIcmt. 
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WORKSHEETS 



Report Period 
YOUTH TREND ANALYSIS 



Complete the bottom portior^ of the worksheet ar^d graph the quarterly/monthly 



trends in performance standards. 


% of Standard 


Period Ending 


Period Ending 


Period Ending 


Period Ending 


150% 
140% 

130% 

120% 

110% 










90% 
80% 
70% 
60% 
50% 
40% 












Performance 
Standards 


Legend 


% of Standard 


Entered Emp. 












Positive Term. 












Cost/Pos. Term. 












Emp. Enhan. 
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WORKSHEET 10 Report Period 

POST PROGRAM TREND ANALYSIS 

Complete the bottom portion of the worksheet and graph the quarterly/monthly 
trends in performance standards. 


"/o of Standard 


Period Ending 


Period Ending 


Period Ending 


PeriodEnding 


150% 
140% 

i3r.: 

120% 
110% 










90% 
80% 
70% 
60% 
50% 
40% 












Performance 
Standards 


Legend 


% of Standard 


F/U Emp. Rate 












F/U Weeks Wkd. 












F/U We!. Emp. 












F/U Wkly. Wage 
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WORKSHEET 1 1 Report Period 

TREND ANALYSIS BY STANDARD 



Select the performance standard, complete the bottom portion of the worksheet and 
graph the results. 

Performance Standard 



% of standard 


Period Ending 


Period Ending 


Period Ending 


Period Ending 


150% 
140% 

130% 

120% 

110% 










90% 
80% 
70% 

WW /O 

50% 
40% 










PY86 EP 
Legend AP 
% 


























PY87 EP 
Legend AP 


















% 










PY88 EP 
Legend AP 


















% 
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WORKSHEET 12 



COMPARISON ANALYSIS 



Report Period 



Fill in the following worksheet using percent of standard accomplished within your SDA (Column A) and other contparable 
SDAs and/or the state as • whole (Columns B-E). Fill in actual values for lines 8-11 (e.g., retention rate 85% = 85% of place- 
ments stiil working after 60 days). 





A 

User SDA 


B 

Other 


c 

Other 


D 

Other 


E 

Other 


1 . Adult Entered Employment Rate 












2. Cost Per Entered Employment 












3. Welfare Entered Employment Rate 
























5. Youth Entered Employment Rate 












6. Youth Positive Termination Rate 












7. Cost Per Positive Termination 












8. Youth Employability Enhan. Rate 












9. Follow-up Employment Rate Adult 












10. ''ollow-up Empl. Rate Adult WIfr. 












11. Follow-up Weeks Worked 












12. Follow-up Weekly Earnings 













Section III: 



Strategies For Improving Performance 



The USDOL has significantly revised the Federal performance 
standards for PY 88. The general objective of the changes in the 
performance standards is to send a clear message to the JTPA 
system that resources and effort should be focused on those 
individuals with multiple barriers to employment and those 
program components which provide long-term benefit to participant 
labor market success. This policy direction supports the 
congressional intent in establishing JTPA. 

For SDAs that need or want to improve performance, there are 
basically two approaches to take. The ^'Oversight Response" option 
helps pinpoint the factors in your existing programs and systems 
which are causing performance problems. The Tlanning Response** 
option helps identify new strategies for improving your perfor- 
mance. Both responses should be viewed as ways of validating or 
modifying the key policies established by the PIC and elected 
officials. 

Oversight Once the key activities causing poor performance have been identi- 

Response fied, the next step is to understand why performance is not meeting 

expectations. The ''Oversight Response** provides the SDA with a 
diagnostic tool to review and identify the factors that contribute to 
poor performance. 

In order for JTPA to function effectively at the local level, a complex 
and interrelated set of program and administrative systems must be 
in place. These systems and procedures cover a broad range, includ- 
ing areas such as participant assessment, job development, contract- 
ing, MIS, supportive services, and classroom training. These 
systems are operated either directly by the administrative entity or, 
in some cases, subccatracted to local service providers. 

These basic elements are classified in the following three categories: 

JTPA Services and Programs - This is the most critical component 
of the employment and training system and includes the following 
activities: outreach, eligibility determination, assessment, employ- 
ability counseling, supportive services (including needs-based pay- 
ments), job development, classroom training, on-the-job training, 
try-out employment, work experience, and other special programs. 

Administrative and Management Systems - The quality and 
effectiveness of JTPA programs and services are directly determined 
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by the basic management and administrative systems which shape 
and support them. The most critical elements here include: 
planning (including youth competencies), organizational structure 
and staffing, fiscal management, MIS, contracting, and monitoring. 

Organizational Roles and Relationships - JTPA allows wide latitude 
in determining the roles and relationships formed in local SDAs. 
The scope and lature of these roles can have a profound influence on 
the quality and cost effectiveness of the programs and services 
provided under JTPA. The critical areas here include: 
r msibilities of the giant recipient and administrative entity, 
planning responsibilities, and cooperative relationships with related 
agencies such as job service, economic development, welfare, and 
other special purpose agencies. 

The Performance Status Reports, Baseline Program Review 
Reports, and Trend and Comparison Analysis Reports detailed in 
Section n can provide you with an abundance of data which offer 
specific "access points" to program areas, contractors, or activities 
with performance problems. Failure to meet a given performance 
standard should be viewed by management as a flag of a possible 
problem. However, tlie performance standards model cannot tell the 
program manager exactly what the problem is. 

Using the observed performance problem as the departure point, the 
Oversight Response enables the SDA to determine which of the 
basic employment and training systems may be causing the 
nerformance problem and provides specific guidance on bcth the 
areas to be reviewed ar. 1 the method of review. 

For example, suppose an SDA's actual cost per entered employment 
is substantially above the standard predicted by the model. The 
manager would first identify programs or contractors significantly 
above the standard. This could be determined from the baseline 
program review conducted in Section n. Assuming that the entered 
employment rate meets the standard, the problem could be isolated 
to the cost of training. The manager might then determine if the 
problem was caused at the planning level, the contracting level, or if 
the actual costs of training differ from planned and contracted 
terms. Thus, a review would be conducted of the planning and 
contracting systems of the SDA and the fiscal system of the specific 
programs or contractors. This oversimplified example does not 
address many important factors, but it does illustrate the type of 
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40 



branching analysis that can pinpoint the cause of performance 
problems. 



Some SDAs may opt for a total review of all the suggested systems in 
order to maximize performance rather than confine the review to 
programs identified as having problems in the baseline review. By 
conducting a complete review of all systems the SDA has an 
opportunity to approach performance improvement in a more 
comprehensive manner, allowing for, in most cases, a one-time 
alignment of these primary systems. 

SDAs should use the Oversight Response Summary to identify the 
program, administrative, and organizational systems that may be 
causing poor performance; the JTPA systems which are most likely 
to influence the standard are marked with an X. Since all systems 
may affect performance standards to some extent, SDAs should be 
familiar with the interrelationship of the standards and concentrate 
their reviews on the primary problems. 

Once an SDA has determined which systems should be examined 
further, it should refer to the Systems Summaries which provide 
guidance on both the areas to be reviewed and the method of review. 
Finally, appropriate corrective action plans should be formulated 
and implemented. 



Summary of 
Oversight 
Response' 
Approach 



Step 1 - Identify specific performance measures needing improve- 
ment through Performance Status Report. 

Step 2 - Identify programs, contractors, or problems wiih target 
groups which may be causing poor p-rformance by using 
Worksheets 4, 5, 6, and 7. Make allowances for varying 
client needs and service strategies. 



Step 3 - Identify probable systems causing poor performance 
either within the program or contractor or throughout 
the entire program using the Oversight Response 
Summary. 

Step 4 - Use the Systems Summaries to conduct a review of each 
area identified above. 

Step 5 - Design, implement, and follow-up on corrective action 
recommendations. 
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OVERSIGHT RESPONSE SUMMARY 




Most likely to influence entered 
employment rates and positive 
termination rates 
System Comments 


Most likely to influence co>i 
standards 

System Comments 


Most likely to influence average 
wage at placement 

System Comments 



JTPA Services & Programs 

Services 
Outreach 

Eligibility Determination 

Assessment 

C6unseling 

Needs Based System 

Other Supportive Services 

Job Development/Placement 

Proarams 

SkillTraining 

Other Classroom Training 

Special Target Programs 

Work Experience 

OJT 



Initial outreach and assessment 
(matchino client to training) 
directly affect dropout rate. 

Ongoing counseling, needs- 
based payments, and other 
supportive services also affect 
retention. The job development 
effort is most citical to these 
standards. 



The quality of training and OJT 
slots directly affects these 
starKiards 



Administration & 

Management Systems 

Planning (including youth 

competencies) 

Organization/Staffing 

FiKal 

MIS 

Contracting 

Monitoring 



Accurate MIS is required, as well 
as clear, performance-type 
contracts, in order to meet 
standards. 
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Oraanizational Rdes & 
Relationships 
PIC-LEO Agreement 
Coordination with: 
Job Service 

Economic Development 
Unemployment Insurance 
Welfare 



I 



i 



x| Cost of needs-based payments 
and Supportive services affects 
cost standards 



I 

JL 



A significant variance in average 
length of training and actual 
cost will affect the cost 
standards 



Job development most 
directly affects this standard. 



The nature and quality of 
training and OJT affect the 
starting wage of completers 



I Planning in demand occupations 
I is essential. 

Status of youth competency 
system will have a dramatic 
effect on positive termination 
rate. 



Inadequate youth competency 
system will affect *cost per' 
standards. 

in ProMems with cost and length 
V of i/ainingmayoccuratthe 
"T" planning, contracting, or 
implementation stage. 



Problems may occur at the 
planning or contracting 
stage. 



Problems at this level could have a negative effect on all performance standards, depending on 
organizational structure^ 



The nature and vitality of coor- 
dination/linkage agreements 
with other agencies can have a 
significant influence on these 
standards 



Cost-effective interagency 

areements can dramatically 
feet thes^ standards. 



Relationships with place- 
ment and job development/ 
creation agencies will affect 
this standard. 
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SYSTEMS SUMMARIES 
PROGRAMS AND SERVICES 



J. OUTREACH 
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AREAS TO BE REVIEWED 

Organization and staffing 
Subrecipients' responsibilities 

Linkages with community agencies serving targeted groups 
Specific outreach goals by target group and geographic area 
Outreach process - use of media, written material, personal contact 

METHOD OF REVIEW 

Review outreach goals by target group 

Interview staff 

Review sources of referrals 

Track characteristics of applicants and enrollees 



ELIGIBILITY DETERMINATION/SELECTION 



AREAS TO BE REVIEWED 

Intake/eligibility process 

Stafi/subrecipient responsibilities 

Length of time from application to enrollment 

Characteristics of eligibles compared to those enrolled 

Existence of specific goals 

METHOD OF REVIEW 

Review the eligibility system as described in JTPA 
Examine application/eligibility determination process 
Analyze characteristics of eligibles not enrolled 



ASSESSMENT 



AREAS TO BE RE VIEWED 

Administration of assessment 
Staffing and organization 
Quality of assessment 

Relevance of assessment data to entrance criteria 

METHOD OF REVIEW 

Review assessment procedures and tools 

Sample assessment files 

Sample employability development plans 

Review skills and characteristics of successful completers 



COUNSELING 



AREAS TO BE REVIEWED 
Organization and staffing 

Procedures for counseling throughout the JTPA system 
Use of the EDP as an ongoing tool 

METHOD OF REVIEW 

Interview clients 

Review client folders for progress reports 
Determine caseload tr'^nds 



31 



5. NEEDS BASED PAYMExNT SYSTEM/SUPPORTIVE SERVICES 



AREAS TO BE REVIEWED 

Methodology for needs-based payment determination 

Consistency of such payments for different target groups and activities 

Effect of payments and services on dropout rates and enrollment trends 

METHOD OF REVIEW 

Review written procedures 

Track dropout rates and underenrollment trends 

6. JOB DEVELOPMENT/PLACEMENT 



AREAS TO BE REVIEWED 

Organization and staffing 
Subrecipient responsibility 
Performance-based contract usage 
Coordination of job development with job service 
Marketing plan and implementation 

METHOD OF REVIEW 

Interview job development and placement staff 

Review job orders and placement documentation 

Track specific vocational training activities with poor wage rates 

7. SKILL TRAINING/OTHER CLASSROOM TRAINING 



AREAS TO BE REVIEWED 

Organization and structure of the course 

Instructional materials 

Staffing 

Instructor's flexibility and responsiveness 
Frequency and accuracy of trainee assessment 

METHOD OF REVIEW 

Review course curriculum 

Conduct site visits 

Interview instructor and trainees 



8. SPECIAL TARGET PRCORAMS 



AREAS TO BE REVIEWED 

Ability of targeted programs to increase participation of the "hard to serve" 
Services and support provided to "hard to serve" populations 

METHOD OF REVIEW 



Perform a desktop review of each targeted program 

Review enrollments, positive terminations and placement data 

Compare enroUee profile to SDA goals for target groups 




9. WORK EXPERIENCE (YOUTH EXEMPLARY PROGRAMS) 

AREAS TO BE REVIEWED 

Organization and staffing 

Characteristics of clients served in this category 

Quality of training experience 

METHOD OF REVIEW 

Review work experience agreements 
Conduct site visits 

Interview SDA staff responsible for monitoring worksites 
Interview participants 
Review timesheets 
Review progress reports 

10. OJT 

AREAS TO BE REVIEWED 
Organization and staffing 

Written procedures describing specific responsibilities 
Linkages with other job development activity at subrecipient level 
Procedures for marketing 
OJT contract development 

METHOD OF REVIEW 

Review OJT contracts 
Conduct site visits 
Review completion rates 



ADMINISTRATION AND MANAGEMENT SYSTEMS 
11. PLANNING 

AREAS TO BE REVIEWED 

Organization and stafHng 
Plan preparation 
PIC involvement 
Planning process 
Youth competency system 

Procedures for tracking performance of subcontractors 

Procedures for collecting and distributing ongoing performance information 

METHOD OF REVIEW 

Review wrii^n procedures 

Interview staff 

Review program evaluations 

Review PIC input through interviews or review of minutes 

Examine youth competency system procedures and monitor implementation 
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12. ORGANIZATION AND STAFFING 

AREAS TO BE REVIEWED 

Number and functions of staff 
Lines of authority 

Formal and informal methods of excharging information 
Staff training and development efforts 
Ratios of service staff to client population 

METHOD OF REVIEW 

Review organizational charts and job descriptions 

Interview staff to validate lines of authoi icy and information exchange procedures 
Examine the reasonableness of agency departments or units 
Anal3rze ratios of staff to clients for program and service systems 



13. FISCAL 



AREAS TO BE REVIEWED 

Staffing and procedures 

Budgeting procedures and cost categories 

Accrual sjrstem, report preparation and distribution 

Subrecipient reporting system 

Timeliness of reports 

METHOD OF REVIEW 

Review back-up of total JTPA budget and subrecipient budgets 

Interview staff, review and validate monthly reports, accruals and report distribution 

14. MIS 



AREAS TO BE REVIEWED 

Organization and staffing 
Data collection 
Expertise of MIS staff 

Distribution of reports and validation process 
METHOD OF REVIEW 
Review written procedures 

Validate written procedures and definitions through interviews with staff 
Track sample reports to source documents 



15. CONTRACTING 



AREAS TO BE REViEWED 

Organization and staffing 
Soliciting proposals, evaluating proposals 
Negotiating contracts, modifying contracts 
Evaluating contracts 

Staff expertise in federal, state, and local procurement regulations 
Clarity of request for proposals 
Specificity of contract goals and objectives 
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METHOD OF REVIEW 



Reviev/ written procedures 
Inter iew staff 

Track RFP process through contract execution and modification 
Review use of performance contracts 

16. MONITORING 



AREAS TO BE REVIEWED 

Organization and staffing 
Monitoring procedures 

Frequency and depth of monitoring and evaluation 
Process for report distribution and corrective action 
Utilization of evaluation results in planning and contract modification 

METHOD OF REVIEW 

Review written procedures 

Interview staff and determine the level and frequency of monitoring 
Track reports through corrective action process 

Determine the extent to which evaluations are used in the planning process 



ORGANIZATION AND COORDINATION 

17. PIC - LEO AGREEMENT 

AREAS TO BE REVIEWED 

Clarity of roles and responsibilities 

Procedures for developing job training plan 

Level of involvement in policy guidance and oversight 

METHOD OF REVIEW 

Review written agreement 
Interview PIC members 
Review PIC meeting minutes 

18. JOB SERVICE 



AREAS TO BE REVIEWED 

SDA/Job Service coordination agreement 
Staffing and organization 
Information exchange 
Client flow 

METHOD OF REVIEW 

Review written agreement 
Interview Job Service and SDA staff 
TvLck number and results of referrals 
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19. ECONOMIC DEVELOPMENT 
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AREAS TO BE REVIEWED 

Cooperative SDA/Economic Development agreements 

Procedures used to share local economic development and labor market information 
Information exchange: 

• Listing of companies planning to relocate to the SDA 

• Listing of companies anticipating expansion within the SDA 

• Listing of planne^i or actual plant closings or reductions in force within the SDA 

METHOD OF REVIEW 
Review written agreements 

Interview staff of economic development agencies and SDA 
Track results of any cooperative projects 

20. UNEMPLOYMENT INSURANCE DIVISION 
AREAS TO BE REVIEWED 

Coordination agreement with the local Unemployment Insurance Division 
Procedures for timely information exchange and referral of UI claimants 

METHOD OF REVIEW 

Review written procedures 

Interview appropriate UI and SDA staff 

Analyze number and results of UI referrals 

21. WELFARE 

AREAS TO BE REVIEWED 

Welfare recipient participation rates 
Information exchange and referrals 

METHOD OF REVIEW 

Review written procedures 

Interview staff of public assistance agencies and SDA 
Track numbers and results of referrals 
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Planning In addition to correcting deficiencies identified through the *'Over- 

Response sight Response,*" most SDAs engage in what is termed the "Planning 

(or Replanning) Response." That is, they rethink key policy 
decisions, after examining the results, and initiate new policies in 
such critical planning areas as: 

• Who will be served 

Job training funds are limited. Only a portion of the eligible 
population can be served in most communities. Beyond the 
Federal requirements to serve particular target groups, SDAs 
decide which groups of individuals will receive priority. 

• What services and programs will be provided 

Taking into account the characteristics of the people who will be 
served, SDAs determine the mix of training, remedial programs, 
and services that will be offered and how financial resources will 
be allocated. 

• What will be acceptable performance standards 

In addition to the minimum standards established by DOL and 
the governor, SDAs may develop additional performance guide- 
lines or request waivers of state standards when appropriate. 

The performance standards system, while establishing accounta- 
bility for such policy and planning decisions, does not usurp this 
important local policy making role. In fact, one of the strengths of 
the performance standards system is that it does not reward or 
penalize SDAs for these types of policy choices, but simply holds 
them accountable for achieving performance standards which reflect 
those choices. SDAs which allow performance standards to **drive" 
the decisions in these key areas rather than consciously establishing 
policies will quickly get lost in a maze of variables and formulas at 
the expense of quality programs. 

Local policy makers should, however, have a clear understanding of 
the mechanics of the performance standards system so that they can 
be aware of the performance-related implications of their decisions. 
The following list shows that 20 of the 23 ac^justment variables 
which affect the computation of an SDA's expected performance are 
directly influenced by the SDA's policy making and planning 
decisions in the areas noted above. 
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Adjustment Variables 
Influenced By SPA 
Who Is Served 



1 

1. 


Female 


2. 


Age 14-15 




Age 16-17 


A 


Age 30 + 


5« 


Black 




Hispanic 


■ 7. 


Asian/Pacific Islander 


8. 


Dropout 


9. 


Student 


10. 


Post High School Attendee 


11. 


AFDC Recipient 


12. 


GA/RCA Recipient 


13. 


Unemployment Compensation Claimant 


14. 


Handicapped 


15. 


Offender 


16. 


Person not in labor force 


17. 


Unemployed 15 + weeks 



What Type of Services and Programs Will be Provided 
18. Percent Terminees 



Not Influenced By SPA 

19. Average Annual Wage in Wholesale/Retail Trade 

20. Unemployment Rate 

21. Population Density 

22. Number of Families below Poverty Level 

23 . Employee/Resident Worker Ratio 



Moreover, as can be seen on the Effect of Changing Variables on 
Performance Standards'" chart, SDAs can determine precisely what 
Impact each of the variables will have on their expected perfor- 
mance. The variables across from each of the performance 
standards are listed in order of their signiflcance. For example, the 
variable which most signiflcantly decreases the adult entered 
employment rate is unemployment rate. The next most influential 
variable for decreasing this standard is GA/RCA Recipients. 
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EFFECT OF CHANGING VARIABLES ON PERFORMANCE STANDARDS - 1988 Model 
(Listed in order of significance of impact) 


Performance Standard 


Raises Standards 
if increased 


Lowers Standards 

if increased 


Entered Employment Rate - 
Adult 

(The hierarchy of variables 
for Welfare Entered 
Employment Rate is the 
same as it is for Entered 
Employment Pate.) 


1. Population Density 


1. Unemployment Rate 

2. GA/RCA Recipient 

3. Dropout 

4. Age 30 and Above 

5. AFDC Recipient 

6. Not in Labor Force 

7. Female 

8. Emp/Res Worker Ratio 

9. Black 

10. Unemployed 15 weeks or + 


Average Wage at Placement 
Adult 


1. Average Annual Earninas 
in Retail/Wholesale Trace 

2. Population Density 

3. U.C. Claimant 

4. Age 30 and Above 


1. Fam. Below Poverty Level 

2. Dropout 

3. Unemployment Rate 

4. Female 

5. Black 

0. cmpi/Kes worKer Katio 
7. Unemployed 15 weeks or -t- 


Entered Employment Rate - 
Youth 




1. Unemployment Rate 

2. 14-15YearsOld 

3. GA/RCA Recipient 

4. Student 

5. Dropout 

6. Handicapped 

7. Black 


Positive Termination Rate- 
Youth 


1. Population Density 

2. Post-H.S. Attendee 

3. Student 


1. Asian/Pacific Islander 

2. GA/RCA Recipient 

3. Hispanic 

4. Empi/Res Worker Ratio 

5. Black 


Employability Enhancement 
Rate - Youth 


1. 14-1 5 Years Old 

2. Student 

3. Dropout 

4. Not in Labor Force 
5 16-17 Years Old 

6. Female 


1. Handicapped 
1 BIcck 

3. Ga7rCA Recipient 

4. AFDC Recipient 


Follow-Up Employment Rate 
-Adult 


1. UC Claimant 


1. Unenfiplqyment Rate 

2. GA/RCA Recipient 

3. Offender 

4. Black 

5. Dropout 

6. Age 30 and Above 

7. AFDC Recipient 

8. Not in Labor Force 

9. Unemployed 15 weeks or -i- 
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EFFECT OF CHANGING VARIABLES ON PERFORMANCE STANDARDS - 1988 Model (cont.) 

(Listed in order of significance of impact) 


Performance Standard 


Raises Standards 
if increased 


Lowers Standards 

if increased 


Follow-Up Weeks Worked - 
Adult 




1. Unemployment Rate 

2. GA/RCA Recipient 

3. UnemployecTlS weeksor + 

4. Black 

5. AFDC Recipient 

6. Dropout 


Follow-Up Weekly Earnings- 
-Adult 


1. Avg. Annual Earnings in 
retail/Wholesale Trade 

2. Population Density 

3. UC Claimant 

4. Age 30 and Above 


1. Unemployment Rate 

2. Fam. Below Poverty Level 

3. Female 

4. Offender 

5. Dropout 

6. Empl/Res Worker Ratio 

7. Unemployed 15 weeksor + 

8. Black 


Raising/lowering the cost standards below has different implications than raising/lowering the eight standards 
listed above. Exceeding your standard for the two cost standards means actual performance lower than the 
standard. 


Cost Per Entered 
E mployment- Ad u It 


1. Population Density 

2. Avg. Annual Earnings in 
Retail/Wholesale Trade 

3. Unemployment Rate 

4. UC Claimant 

5. AFDC Recipient 

6. GA/RCA Recipient 

7. Not in Labor Force 

8. Hispanic 


1. %Terminees 


Cost Per Positive 
Termination - Youth 


1. Avg. Annual Earnings in 
Retail/Wholesale Trade 

2. AFDC Recipient 

3. Hispanic 

4. Handicapped 

5. Not in Labor Force 


1. %Terminees 

2. Student 

3. 14-1 5 Years Old 
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It is important to note that the impact of specific variations on the 
overall standard is not, in most cases, substantial. The cumulative 
effect, however, can significantly change the local SDA standards. 



Familiarity with the dynamics of the performance standards system 
will enable SDA stafi* to examine how lociA factors are shaping 
currtr ^ expected performance and to determine what efiect program 
modifications would have on future expected performance levels. By 
following the "exam: ing impact" instructions on the next page and 
using Worksheets 1^ and 13, an SDA can identify the extent to 
which each factor is increasing or decreasing their expected 
performance levels. Users should also compare local values with 
extreme factor values detailed in Section G of the Department of 
Labor Guide For Se t ting JTPA Title IhA Performance 
Standards For FY bS. The value of this exercise is in 
understanding how local factors have shaped current expected 
performance levels, not in determining the extent to which the SDA 
has deviated from the national averages. 

Once an SDA has completed the type of analysis recommended in 
this guide, it may consider changes in program design or emphasis 
for the balance of the program year. The Baseline Program Review 
conducted in Section II provides valuable information for this 
replanning effort. Obviously any new program emphasis should be 
based on what has worked best, for what groups, and at what cost. 
By following the "dry run" instructions on the next page and using 
Worksheets 13, 14, 15 a^d 16, SDAs will be able to determine 
precisely what efiect these various policy and planning options 
would have on their expected performance levels. 
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INSTRUCTIONS FOR WORKSHEETS 13, 14, 15 AND 16 



Option A - Recalculation Based on Actual Terminee Characteristics 

This option enables SDAs .o recalculate their expected performance 
standards based on the actual terminee characteristics. 

Complete only Columns B, D, E, and F of the top section and Columns 
A, B, and C of the bottom section* 

Option B - Examining Impact of Liocal Factors on Expected Performance Levels 

This option enables SDAs to determine the extent to which local factors 
have increased or decreased expected performance levels. (Examine 
Colunms D, E, and F to observe + or- effect of local factors on current 
expected performance levels.) 

Complete only Colunms B, D, E, and F of the top section. 

Option C - Dry Run of Varic — Policy and Planning Options 

This option enables the SDA to determine precisely what effect various 
policy and planning options would have on the expected perfomiance. 

Complete only Colunms B, D, E, and F of the top section and Column B 
in the bottom section. 

f 

Description of top section columns 

Column A. National average percent of terminees 

Column B Actual percent of terminees at the time of review 

Columr C Percent of national average 

Column D. Calculates the effect on each standard indicated within 
the column (Bl) actual terminee percent minus (Al) 
national standard times (.20) weight = effect 

Column E. Same as above 

Column F. Same as above 

Dt^scription of bottom section columns 

Column A. Actual local measures 

Column B. Actual standards if calculated on actual current terminees 
(The SDA must add the effects from each column above 
and a4]ust the national standard to display the local model 
adjusted standard) 

Column C. Percent of standard 
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Workshecftia 



MODEL ADJUSTEr WORKSHEET FOR ANALYZING CLIENT CHARACTERISTIC PERFORMANCE - ADULT - PY 88 


Variables 


National 


1 Actual 


Percent 


Standards Calculations 




A 

A. 


D 
D. 


c 


D. 


E. 


F. 


Adult 






[B/A] 


Fnt Fmn Ratp 
kilt. r\aic 




^Vl9i/Clil. CiTip. 




Mveroge wage 




1. Female 


55.1 






fDi A i/ n70\1 




llllllhllllllllllllllllll 




[B1-A1(-.0065)] 




2. Age 30 + 


52.8 






fD^ A^/ iCC\1 




lllllllllllllllllillllllU 




[B2*A2(.0033)] 




3. Black 


23.2 






Tdo AO/ ncc\i 




llllllllllllllllllllllllll 




[B3-A3(-.0051)] 




4. Hispanic 


8.3 






////////////////////////// 
llllllllllllllllllllllllll 




1B4-A4(6.2)J 




ttllllHlllltlllllllllllllll 

llllllllllllllllllllllllllll 




5. Dropout 


24.8 






TDC AC/ 1^'7\1 




llllllllllllllllllllllllll 




[B5-A5(-.0119)] 




6. UC Claimant 


10.3 






llllllllllllllllllllllllll 




[B6-A6(34.5)] 




[B6*A6(.0290)| 




7. Unempl. 15 + 


48.7 






rB7-A7(- 01511 




lllllllllllllllllillllllU 




rR7-A7/- 00101 




8. Not in Labor F 


11.9 






[B8-A8(-.082)] 




[B8-A8(8.0)] 




llllllllllllllllllllllllllll 




9. AFDC 


23.8 






[B9-A9(-.159)] 




[B9-A9(24.9)] 




llllllllllllllllllllllllllll 




10. G A/RCA 


5.2 






[B10-A10(-.312)] 




[BIO-AIOd 5.2)1 




llllllllllllllllllllllllllll 




11. %Terminees 


73.2 






llllllllllllllllllllllllll 




[B11-A11(-22.0)] 




llllllllllllllllllllllllllll 




12. Unemp. Rate 


7.4 






[B12-A12(-.608)] 




[B12-A 12(63.8)] 




[B12-A12(-.0089) 




13 Avg. EarnR/W 


12.5 






llllllllllllllllllllllllll 




[B13-A 13(79.0)] 




[B13-A13(.0936) 




14. Pop. Density 


0.7 






[B14-A14(.633)] 




[B14-A14(79.5)] 




[B14-A14(.0880)] 




15. Fam.Bel.Pov. 


9.7 






//////////////////////////// 




llllllllllllllllllllllllllll 




[B15-A15(-.0307) 




16. Emp. Res Ratio 


99.9 






[B16-A16(-.064)] 




lllllllllllllllllllllllllll 




[B16-A16(-.0040) 





NET EFFECT NET EFFECT NET EFFECT 

Summary Section 



Performance Standard 


Calculated Standard 


Actual Performance 


% of Standard 


Entered Employment Rate 


"D" Net Effect = 


-1- 68.0 






Cost per Entered Employment 


"E" Net Effect 


-1- $4500 






Welfare Entered Employment 


"D" Net Effect = 


+ 68.0 (x)* = 






Wage at Placement 


"F" Net Effect = 


+ $4.95 
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Worksheet 14 

MODEL ADJUSTMENT WORKSHEET FOR ANALYZING CLIENT CHARACTERISTIC PERFORMANCE - YOUTH - PY 88 



Variables 
Touxn 


Nat'l. Actual Pet. 


Standards Calculations 


A. 


B. 


C. 


D. 


E. 


F. 


G. 






fQ/Al 

[B/Aj 


Ent EmD Rate 




Term R;it0 

r\/9. 1 Villi. r\OiV 




Ffvm Cnl^ DatA 

cmp. cnn. noXc 




V>0>wrO>. Icrfil. 




1. Female 


50.6 






llllllllllllllllllllllll 




llllllllllllllllllllllll 




[B1-A1( + .064)1 




llllllllllllllllllllllll 




2. Age 14-15 


5.7 






[B2-A2(-. 589)1 




llllllllllllllllllllllll 




[B2-A2( + .332)1 




[B2-A2(~9.5)) 




3. Age 16-17 


32.4 






llllllllllllllllllllllll 




llllllllllllllllllllllll 




[B3-A3( + . 070)1 




llllllllllllllllllllllll 




4. Black 


27.9 






[B4-A4(-.058)] 




[B4-A4(-.045)l 




[B4-A4( -.071)1 




llllllllllllllllllllllll 




5. Hispanic 


9.5 






llllllllllllllllllllllll 




[B5-A5(-. 060)1 




llllllllllllllllllllllll 




[B5-A5( + 9.0)1 




6. Asian /Pacific Islander 


2.0 






llllllllllllllllllllllll 




[B6-A6(-. 122)1 




llllllllllllllllllllllll 




llllllllllllllllllllllll 




7. Dropout 


25.2 






[a7-A7(-.273)l 




llllllllllllllllllllllll 




[B7-A7( + .139)1 




llllllllllllllllllllllll 




8. Student 


39.6 






[B8-A8(-.3e0)] 




[B8-A8(^. 027)1 




[B8-A8( + . 253)1 




[B8-A8<- 11.7)1 




9. Post High School 


6.7 






llllllllllllllllllllllll 




[B9-A9(^. 202)1 




llllllllllllllllllllllll 




llllllllllllllllllllllll 




10. Handicapped 


15.9 






[B10-A10(-.068)] 




llllllllllllllllllllllll 




[B10-A10(-.1031 




[B10-A10( + 8.4)1 




11. Not in Labor Force 


38.7 






llllllllllllllllllllllll 




llllllllllllllllllllllll 




IB11-A11( + .091)1 




[B11-A11( + 4.2)1 




12. AFOC Recipient 


20.4 






llllllllllllllllllllllll 




llllllllllllllllllllllll 




[B12-A12(-.035)l 




[B12-A12(+ 15.1)1 




13. GA/ RCA Recipient 


3.3 






[Bl3-Al3(-.370)) 




[B13-A13(-. 116)1 




[B13-Al3(-.067)] 




llllllllllllllllllllllll 




14. %Terminees 


75.9 






llllllllllllllllllllllll 




llllllllllllllllllllllll 




llllllllllllllllllllllll 




[B14-A14(-18.5)l 




15. Unemployment Rate 


7.4 






[B15-B15(- 1.333)1 




llllllllllllllllllllllll 




llllllllllllllllllllllll 




llllllllllllllllllllllll 




16. Population Density 


.7 






llllllllllllllllllllllll 




[B16-A16(, 1.353)1 




llllllllllllllllllllllll 




llllllllllllllllllllllll 




17. Emp. Res/ Ratio 


99.9 






llllllllllllllllllllllll 




[B17-A17(-051)] 




llllllllllllllllllllllll 




llllllllllllllllllllllll 




18. Avg. Earnings- W/R 


12.5 






llllllllllllllllllllllll 




llllllllllllllllllllllll 




llllllllllllllllllllllll 




[Bl8-A18( + 64.3)] 





NETEFFEa NET EFFECT NET EFFECT NETEFFEa 



Summary Section 



Performance Standard 


Calculated Standard 


Actual Performance 


% of Standard 


Entered Employment Rate 


-D- Net Effect = ♦ 45.0 = 






Positive Termination Rate 


"E" Net Effect = ♦ 75.0 = 






Employability Enhancement Rate 


•□"Net Effect = ♦ 30.0 = 






j Cost per Positive Termination 


"F* Net Effect = ♦ $4900 = 
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Worksheet IS 



MODEL ADJUSTED WORKSHEET FOR ANALYZING CLIENT CHARACTERISTIC PERFORMANCE 

ADULT POST PROGRAM • PY 88 



Variables 


National 


Actual 


Percent 


Standards Calculations 




A. 


B. 


C. 


D. 


E. 


F. 


Adult 






IB/AJ 


FAJ Emp Rate 




F/U Weeks Work 








1. Female 


55 1 






//////////// //////// ////// 




ttttttttllttlM. '§!§§§§§§§§ 




IB1-A1(-.511)1 




2. Age 30 


52.8 










IIIIIIIIIIIIIIIIIIIIIIIIII 
IIIIIIIIIIIIIIIIIIIIIIIIII 




IB2-A2(.488)J 




3. Black 


23.2 










1D3-A3(-.019)J 




1B3-A3(-.081)] 




4. Dropout 


248 






f Dil Ail/ 




|B4-A4(-.013)] 




(B4-A4( .360)1 




5. UCCIaimar)t 


10.3 






|B5-A5(.123)1 




IIIIIIIIIIIIIIIIIIIIIIIIII 




(B5-A5(1.516)1 




6. Offender 


8.4 






(B6-A6(-.131)] 




IIIIIIIIIIIIIIIIIIIIIIIIII 




(B6-A6( .361)1 




7. Unempl. 15-f- 


48.7 






|B7-A7(-.038)) 




IB7-A7( .016)1 




IB7-A7( .097)j 




8. Not in Labor F | ii 9 






|B8-A8(-.049)) 




IIIIIIIIIIIIIIIIIIIIIIIIII 




llllllllllllllllllllllllllll 




9. AFDC ' 


23.8 






(B9-A9( .109)] 




(B9-A9( .014)1 




llllllllllllllllllllllllllll 




10. GA/RCA 


5.2 






(B10 A10( .165)] 




(B10A10(-.023)1 




llllllllllllllllllllllllllll 




11. Unemp. Rate 


7.4 






(B11-A11(-.860)] 




(B 11 -A 11 (-.060)1 




IB11-A11(-1.167) 




12 Avg.EarnR/W 


12.5 






nnnnnnnininnnn 




llllllllllllllllllllllllllll 




(B12-A12(2 867) 




13. Pop. Density 


0.7 






iiiiiiiiiiiiiiiiiiiiiiiiii 




llllllllllllllllllllllllllll 




(B13-A1 3(2.248) 




14. Fam. Bel. Pov. 


9.7 






iiiiiiiiiiiimiiiiiiii! 




llllllllllllllllllllllllllll 




(B14-A14(-.762)l 




1 5. Emp. Res Ratio 


999 






iiiiiiiiiiniiinnininin 




llllllllllllllllllllllllllll 


|(B15-A1S(-.222) 




NET EFFECT NET EFFECT NET EFFECT 

Summary Section 



Performance Standard 



Calculated Standard 



Actual Performance 



% of Standard 



Follow-Up Employment Rate 



•D- Net Effect = 



60.0 



Follow-Up Weeks Worked 



•E" Net Effect s 



^ 80 



Follow-Up Weekly Earnings 



•F" Net Effect 



+ $177 



ERIC 



9^- F/U Welfare Employment Rate 



'D" Net Effect 



+ 60.0 (x)* = 



(x) K Public Assistance Employment Ratio ( Method II ) 
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Worksheet 16 



MODEL ADJUSTED WORKSHEET FOR ANALYZING CLIENT CHARACTERISTIC PERFORMANCZ 

ADULT WELFARE - PY88 


Variables 


National 


Actual 


Percent 


Standartls Calculations 




A. 


8. 


C. 


D. 


E. 


All Adult Welfare 






[B/A] 


Ent. Emp. Rate 




F/U Emp. Rate. 




1. Black 


32.6 






[B1-A1( .046)] 




[B1-A1( .126)] 




2. Dropout 


29.9 






(B2-A2(.155)J 




[B2-A2( .211)] 




3. Unemployed 15 -i- weeks 


54.8 






[63-A3( .068)] 




[B3-A3( .026)] 




4. Not in Labor Force 


19.0 






[B4-A4( .151)] 




[64-A4( .085)] 




5. GA/RCA 


13.9 






[B5-A5( .139)] 




[35-A5( .089)] 




6. Unemployment Rate 


7.4 






[B6-A6(-.721)] 




[B6-A6( .928)] 





NET EFFECT NET EFFECT 



Summary Section 



Performance Standard 


Calculated Standard 


Actual Performance 


% of Standard 


Entered Employment Rate 


•D* Net Effect = 4 56.0 






Follow-Up Employment Rate 


"E* Net Effect = 4 50.0 
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Section IV: 



Conclusion 



The Job Training Partnership Act emphasizes the importance of 
program performance and outcorues. The law establishes specific 
criteria by which an SDA will be measured and provides for the 
award of incentives for local programs that perform exceptionally 
well. The purpose is to ensure that SDAs concentrate on results 
rather than on regulatory and compliance issues. 

Performance standards are not **the'' solution to effective JTPA 
planning and management but rather one of several tools available 
to administrators to improve performance. The challenge to local 
policy makers is to use performance standards to guide the develop- 
ment of programs which will meet the needs of the eligible 
population and business community. 

In order to work effectively with performance standards, an SDA 
must have a clear vision of its role in the community. The key policy 
issues related to who is served and with what type of programs 
should be important building blocks in establishing the foundation 
for planning and management decisions. Since the performance 
standards model adjusts an SDA's standards primarily according to 
wh- is served and the program design, the degree of difficulty in 
meeting the standards will be the same, regardless of who is served 
and what is done for them. For local policy makers who fail to 
establish this foundation, performance standards can easily become 
a **n umbers game," unrelated to the mission and purpose of JTPA in 
the community. 

As JTPA matures, the performance standards system will continue 
to change. New measures will be added, standards will be modified, 
the a4justment methodology will be revised, and so on. However, 
the need to understand the dynamics of the performance standards 
system and its relationship to planning and management will 
remain constant. SDAs need to stay abreast of such changes and 
update and refine their planning and management practices as 
changes occur. 
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National Alliance of Bu 



New England 

Connecticut, Maine, Massachusetts, 
New Hampshire, Rhode Island, 
Vermont 

Joseph Fischer, Regional Vice 

President 

20 Walnut Street 

Wellesley Hills, Massachusetts 02181 
(617)235-1332 



Atlantic 



Delaware, District of Columbia, 
Maryland, New Jersey, New York, 
Pennsylvania, Puerto Rico, Virgin 
Islands, Virginia, West Virginia 

Fredrick Radlmann 

Regional Vice President 

317 Gecrge Street 

New Brunswick, New Jersey 08901 

(201)524-4110 



Southeast 



Alabama, Florida, Georgia, Kentucky, 
Mississippi, North Carolina, South 
Carolina, Tennessee 

William 'Bonn/ Walker 

Regional Vice President 

100 Edgewood Avenue, Suite 1800 

Atlanta, Georgia 30303 

(404) 522-9350 



Midwest 



Illinois, Indiana, Iowa, Michigan, 
Minnesota, North Dakota, Ohio, South 
Dakota, Wisconsin 



ERIC 



Regional Service Offices 



Midwest eont*d 



Vincent Serritella 
Regional Vice President 
Eleven East Adams, Suite 1008 
Chicago, Illinois 60603 
(312)341-9766 



Central 



Arkansas, , Kansas, Louisiana, 
Missouri, Nebraska, New Mexico, 
Oklahoma, Texas 

Henry McHenry 

Regional Vice President 

9400 North Central Expressway 

Dallas, Texas 75231 

(214) 373-0854 



West 



Arizona, California, Colorado, Guam, 
Hawaii, Nevada, Pacific Islands, Utah, 
Wyoming 

Janet Keyes 

Regional Vice President 

350 Sansome St. Suite 1040 

San Francisco, California 94104 

(415)391-4061 



Pacific Northwest 



Alaska, Idaho, Montana, Oregon, 
Washington 

Rueben Flores, Regional Vice 
President 

427 Skinner Building 
Seattle, Washington, 98101 
(206) 622-2531 
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